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1.  ABSTRACT 

The  ongoing  research  investigates  the  representations  of  visual  texture  and  the  processes 
that  detect  discontinuities  and  structure  in  visual  texture.  Psychophysical  experiments 
have  investigated  the  salience  of  bar  orientation  and  the  effect  of  groupings  in  texture 
segmentation.  We  are  examining  the  role  of  elongated  receptive  field  mechanisms  in 
computing  both  local  measures  of  orientation  and  their  possible  role  in  texture  segmenta- 
tion.  We  have  found  such  mechanisms,  however,  to  be  less  appropriate  for  determining 
one-dimensional  groupings  of  (collinear)  discrete  items  of  texture.  Combined  psychophy¬ 
sical  and  computational  studies  have  provided  evidence  for  place  tokens  in  groupings, 
and  current  work  is  directed  towards  understanding  how  these  tokens  may  be  defined  in 
fine-scale  texture  detail.  To  support  this  work,  a  vision  laboratory  has  been  established 
based  on  a  Symbolics  3800  Lisp  Machine. 
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found  such  mechanisms,  however  to  be  less  appropriate  for  determining 
one-dimensional  groupings  of  (collinear)  discrete  items  of  texture. 
Combined  psychophysical  and  computational  studies  have  provided  evidence 
for  place  tokens  in  groupings,  and  current  work  is  directed  towards 
understanding  how  these  tokens  may  be  defined  in  fine-scale  texture 
detail.  To  support  this  work,  a  vision  laboratory  has  been  established 
based  on  a  Symbolics  3600  Lisp  Machine. 
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The  ongoing  research  investigates  the  representations  of  visual  texture  and  the  processes 
that  detect  discontinuities  and  structure  in  visual  texture.  The  research  seeks  to  under¬ 
stand  how  the  visual  attributes  of  texture  are  computed  by  the  human  visual  system.  In 
general,  our  approach  is  concerned  with  two  issues,  one  being  how  properties  or  attri¬ 
butes  of  texture  are  defined  (such  as  the  orientation  of  small  blobs  and  bars)  and 
secondly,  how  these  properties  are  used  by  subsequent  processes  (such  as  those  that 
detecting  texture  boundaries  and  detecting  collinear  or  parallel  arrangements  among 
these  elements).  Psychophysical  experiments  are  planned  to  probe  both  the  processes 
that  operate  on  texture,  to  find  boundaries  and  so  forth,  and  the  processes  that  deliver 
the  underlying  visual  representation  of  the  texture.  In  parallel  we  are  developing  a  com¬ 
putational  basis  for  these  processes  that  is  consistent  with  the  known  neurophysiology. 

3.  RESEARCH  STATUS 

In  our  first  year  we  have  acquired  most  of  the  equipment  in  our  computational  vision 
laboratory,  and  begun  several  parallel  experimental  and  computational  studies. 

3.1  Establishing  the  vision  laboratory 

The  laboratory  is  centered  on  a  Symbolics  3000  Lisp  Machine,  which  has  been  installed  >0t 
the  Psychology  Department.  Funding  for  the  equipment  was  received  in  August  1983 
and  delivery  of  the  Lisp  Machine  was  made  in  October.  The  Lisp  Machine  was  con¬ 
nected  via  Ethernet  to  the  Department  of  Computer  and  Information  Science  VAXs  and 
Lisp  Machines  in  December,  thereby  providing  access  to  file  structures  at  the  two  sites.  , 


Auxiliary  items  of  equipment  arc  a  Tektronix  690SR  color  monitor  and  an  NEC  Spin- 
writer.  Also,  a  grant  application  to  Tektronix  for  a  video  hard  copy  device  was 
approved;  a  4634  valued  at  $7900  was  donated  for  our  research  project. 

The  laboratory  is  still  in  its  initial  development,  awaiting  delivery  of  the  Robotics 
Systems  digital  frame  buffer  and  digital  convolution  systems.  This  equipment  is  based 
on  very  new  technology  and  has  involved  development  at  Robotic  Systems  of  a  new 
interface  based  on  the  recently  developed  3600  architecture.  The  delay  in  acquisition  of 
our  color  frame  buffer  has  prevented  our  use  of  the  Lisp  Machine  for  grey-level  and  color 
graphics.  In  the  interim  period  we  have  concentrated  on  experiments  that  can  be  per¬ 
formed  using  the  existing  high  resolution  bit-map  graphics  on  the  3600  console.  We 
have  just  received  the  frame  buffer  and  it  is  currently  being  installed  concurrent  with 
Symbolics’  new  Release  5.0.  The  convolution  system  is  expected  in  June.  When  this 
equipment  is  also  installed  we  will  have  facility  for  digital  convolution  and  image  pro¬ 
cessing  similar  to  that  at  the  MIT  Artificial  Intelligence  laboratory. 

3.2  Research 

Previous  research  has  demonstrated  the  importance  of  a  small  number  of  properties  for 
texture  perception,  including  orientation,  brightness,  and  site.  The  current  research  is 
investigating  in  detail  the  means  by  which  orientation  is  defined  in  texture,  and  the  pre¬ 
cise  role  orientation  plays  in  texture  discrimination,  perceptual  grouping  of  texture  ele¬ 
ments,  and  other  processes  in  texture  perception. 

8.2.1  The  » alienee  of  bar  orientation 

Two  regions  of  texture  composed  of  oriented  elements  will  segregate  on  the  basis  of 


orientation  differences.  Our  research  has  shovn  that  not  all  geometrically  equivalent 
changes  in  orientation  are  equally  effective  in  producing  texture  segmentation.  We  have 
been  developing  a  computational  model  which  allows  us  to  predict  the  effectiveness  of 
orientation  differences  in  producing  texture  segmentation.  One  model  we  currently 
explore  is  based  on  the  assumption  that  an  orientation  is  measured  in  terms  of  elongated 
receptive  field  structures,  and  that  two  regions  of  texture  segment  when  receptive  fields 
of  common  orientation  and  spatial  extent  in  the  two  regions  receive  different  excitations. 
The  degree  of  segmentation  is  weakened  by  the  degree  to  which  texture  elements  in  the 
two  regions  stimulate  receptive  fields  of  common  orientation. 

In  one  experiment,  texture  segmentation  was  examined  in  patterns  consisting  of 
discrete  bars  where  two  regions  differ  in  the  orientation  of  the  constituent  bars.  The 
bars  in  one  region  were  oriented  10  0  from  the  horizontal  and  those  in  the  other  region 
were  either  20,  30,  or  40 0 .  Four  conditions  were  examined  —  corresponding  to  two  bar 
widths  (approximately  V  and  2.5')  and  two  bar  lengths  (approximately  3'  and  6'). 
When  the  orientation  difference  was  20  or  30° ,  increasing  the  width  of  the  6'  -long  bars 
increased  the  perceived  texture  segmentation.  When  the  orientation  difference  was  only 
10  ° ,  increasing  their  width  decreased  the  perceived  texture  segmentation.  When,  on  the 
other  hand,  the  bars  were  3'  in  length,  the  texture  segmentation  decreased  with  the 
wider  bars.  (Except,  for  the  short  bars  a  40°  difference  was  also  examined,  with  the 
result  being  equal  texture  segmentation  for  the  two  widths.)  From  the  point  of  view  of 
stimulating  the  excitatory  region  of  a  elongated  receptive  field,  the  wide  bars  would  be 
much  more  effective  than  the  narrow;  the  visual  system  is  approximately  28  times  more 
sensitive  to  the  wide  than  to  the  narrow.  The  orientation  "difference  signal”  is 
apparently  not  only  a  function  of  the  strength  of  stimulation  of  receptive  fields  of  the 
specific  orientation,  but  also  of  the  number  of  receptive  fields  stimulated  in  common. 


The  smaller  the  angle  difference  and  the  shorter  the  bar,  the  more  two  elements  will 
stimulate  differently-oriented  receptive  fields  similarly,  and  the  less  pronounced  would  be 
the  perceived  segmentation.  One  can  see  that  the  orientation  difference  between  10  and 
20  ’  narrow  bars  is  the  same  as  between  10  and  20  °  wide  bars.  This  suggests  that  the 
geometric  difference  is  not  what  matters,  but  rather  some  measure  that  confounds  orien¬ 
tation  and  energy. 

We  are  presently  investigating  more  carefully  how  texture  segmentation  varies  with 
bar  length,  width  and  intensity,  and  comparing  the  results  to  an  emerging  theory  for  the 
role  of  elongated  receptive  fields  in  the  measurement  of  orientation  differences  in  texture. 
While  we  are  examining  several  computational  models  involving  receptive  field  organiza¬ 
tions,  we  do  not  expect  them  necessarily  to  correspond  to  the  simple  cells  discovered  by 
neurophysiologists.  Other  experiments  (see  below)  are,  in  fact,  implicating  a  higher-level 
encoding  of  the  spatial  properties  of  the  texture. 

3.2.2  The  role  of  groupings  in  texture  segmentation 

Texture  segmentation  cannot  be  accounted  for  solely  in  terms  of  first  order  statistics  of 
the  image  intensity  distribution.  In  addition  to  processes  of  segregation,  the  perception 
of  texture  involve  processes  of  aggregation.  For  instance,  preattentive  grouping  of  ele¬ 
ments  may  give  rise  to  different  texture  descriptions  in  two  abutting  regions.  That  is, 
differences  in  the  resulting  groupings  can  be  shown  to  give  rise  to  perceptually  distinct 
texture  regions.  We  have  begun  experiments  to  investigate  how  element  orientation, 
size,  and  brightness  affect  grouping  processes  in  texture. 


One  type  of  stimulus  consisted  of  three  regions,  a  horizontal  central  region  flanked 
by  regions  above  and  below.  Each  pattern  consisted  of  a  given  type  of  element  (either 
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vertical  bars,  squares  or  circles)  of  two  different  sizes.  In  the  top  and  bottom  regions, 
the  pattern  consisted  of  columns  of  either  large-  or  small-sized  elements,  while  in  the 
middle  region  the  large  and  small  elements  alternated.  The  middle  region  appeared 
more  or  less  distinct  from  the  flanking  top  and  bottom  regions. 

Our  experiments  have  shown  that  a  scaling  of  the  entire  stimulus  patterns  reduces 
texture  segmentation  for  non-oriented  elements  such  as  circles  and  squares,  but  that 
scaling  leaves  texture  segmentation  either  unchanged  or  improved  for  oriented  bars.  A 
second  series  of  experiments  has  shown  that  the  texture  segmentation  is  a  U-shaped 
function  of  intensity.  The  segmentation  is  decreased  when  the  intensity  of  the  (non- 
oriented)  elements  is  increased  from  approximately  1.5  to  6.4  foot-Lamberts.  The  results 
suggest  that  we  are  dealing  with  an  interference  effect.  Increasing  either  the  intensity  or 
the  overall  size  of  the  elements  through  scaling  seems  to  cause  them  to  interfere  with  the 
original  groupings  of  the  elements  into  columns.  An  overall  increase  in  intensity  seem¬ 
ingly  increases  the  salience  of  the  smaller  elements,  putting  them  into  competition  with 
those  in  the  columnar  groupings.  Overall,  this  experimental  series  appears  to  be  reveal¬ 
ing  two  facts:  that  groupings  can  play  a  significant  role  in  texture  segmentation,  and 
that  in  this  role,  oriented  elements  are  functionally  different  from  unoriented  elements. 
These  results  are  not  easily  accounted  for  by  simple  cell-like  feature  detectors  responding 
directly  to  the  columns  of  elements  in  the  top  and  bottom  regions  of  the  patterns.  Sim¬ 
ple  cell-like  detectors  would  be  expected  to  scale,  that  is,  to  maintain  an  approximately 
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constant  proportion  of  length  to  width.  Consequently  the  expectation  would  be  that 
scaling  would  yield  the  same  texture  segmentation.  It  is  also  not  clear  why  an  increase 
in  overall  brightness  would  decrease  the  texture  segmentation. 


8.2.8  Evidence  for  place  tokens 


Visual  texture  is  composed  of  seemingly  discrete  items  or  elements,  even  when  the  under¬ 
lying  intensity  distribution  is  continuous.  One  can  localize  and  define  the  shape  of 
blotches,  bars,  tiny  blobs,  and  so  forth  in  the  image;  when  these  items  are  elongated 
they  are  attributed  an  orientation.  It  is  not  yet  clear  whether  the  orientation  measure 
that  is  attributed  to  discrete  and  visually  distinct  items  in  texture  is  also  that  which 
governs  texture  segmentation  and  other  texture  phenomena,  or  whether  a  more  primitive 
measure  of  orientation  is  responsible  (such  as  a  Fourier-based  or  simple  cell-like  local 
measurement  of  the  intensity  distribution).  Determining  the  answer  to  this  problem  is 
quite  challenging;  to  do  so  requires  developing  specific  models  for  the  two  alteratives. 

Part  of  our  research  on  this  problem  has  involved  finding  evidence  for  texture 
groupings  that  cannot  be  easily  accounted  for  in  terms  of  Fourier-  or  simple  cell-like 
receptive  field  schemes.  To  argue  that  a  given  texture  effect  is  not  accountable  by 
Fourier-based  means  is  an  indirect  argument  for  some  (as  yet  unspecified)  symbolic 
scheme,  one  which  treats  the  individual  elements  as  discrete  items.  Since  it  is  not  clear 
precisely  what  it  means  to  treat  items  of  texture  as  discrete  symbolic  markers  for  group¬ 
ing  purposes,  it  is  still  a  rather  difficult  notion  to  test.  Moreover,  since  there  is  no  com¬ 
mon  understanding  of  how  the  alternative  (Fourier- based)  schemes  are  implemented  in 
human  vision,  similar  difficulties  arise.  Nonetheless,  that  is  the  state  of  affairs  in  this 
area. 

Consider  the  simple  grouping  phenomenon  in  which  a  row  of  dots  appears  to  cohere 
into  a  dotted  line,  a  one-dimensional  grouping.  The  two  alternative  hypotheses  are  i) 
that  the  dots  are  treated  as  place  tokens,  and  ii)  that  they  are  not,  but  that  the  orienta¬ 
tion  of  the  dotted  line  emerges  by  some  spatial  blurring  operation  that  detects  the  linear 


We  have  found  several  types  of  evidence  that  simple  cell  mechanisms  are  not  the 
sole  means  for  detecting  linear  organization  in  dot  patterns.  The  conclusion  we  draw, 
therefore,  is  that  for  at  least  some  one-dimensional  grouping  phenomena,  the  items  that 
comprise  the  perceived  organization  must  be  treated  as  individuals,  as  place  tokens. 
Several  observations  support  this  conclusion. 

First,  we  examined  dot  patterns  that  present  obvious  geometric  organization,  but 
for  which  little  information  is  carried  by  relatively  low  spatial  frequencies.  One  way  is 
to  make  patterns  composed,  not  of  simple  dots,  but  of  small  items  that  present  little 
power  in  the  spatial  frequencies  on  the  order  of  1/s,  where  s  is  the  spacing  between  the 
items.  A  second  line  of  evidence  against  the  Fourier-based  schemes  involves  dot  patterns 
where  the  collinear  dots  are  not  relatively  isolated,  but  rather,  embedded  in  a  roughly 
uniform  and  isotropic  dot  distribution.  For  these  patterns,  if  a  simple  cell  were 
presented  with  a  row  of  dots  in  its  excitatory  region,  there  would  be  roughly  equal 
energy  presented  in  its  inhibitory  flanks,  and  consequently  the  linear  organization  among 
the  dots  would  not  be  detected.  An  example  of  this  sort  of  pattern  is  given  by  the  Mar- 
roquin  patterns.  F urther  evidence  is  provided  by  the  simple  observation  that  we  can  see 
pairings  between  dots  that  are  separated  by  approximately  1  ° ,  which  is  well  beyond  the 
largest  sustained  channel  (8.8')  in  the  central  fovea.  Still  more  evidence  against  a 
Fourier-based  approach  is  posed  by  rivalrous  dot  patterns  in  which  two  potential  organi¬ 
zations  could  be  imposed,  one  between  dim  dots,  another  between  dim  and  bright  dots. 
The  simple-cell  model  would  predict  the  pairings  between  dim  and  bright  dots;  the  per¬ 
ceived  organization,  however,  is  between  the  similar,  but  dimmer,  pairs  of  dots. 


The  various  demonstrations  together  argue  against  the  specific  proposal  that  bar¬ 
like  simple  cells  are  wholly  responsible  for  the  detection  of  linear  organization  in  dot  pat¬ 
terns.  The  more  fundamental  argument,  however,  is  against  the  sufficiency  of  local 
Fourier  analysis  for  detecting  collinearity  among  point-like  features  in  an  image.  The 
alternative,  as  we  see  it,  is  that  the  dots  are  treated  by  the  visual  system  as  discrete 
items,  and  their  collinearity  noticed  by  geometric  constructions  among  these  discrete 
items.  The  basic  issue  is  the  localization  of  the  items  that  constitute  one-dimensional 
groupings.  We  have  shown  that  it  is  not  sufficient  to  employ  techinques  based  on  spatial 
blurring;  instead  it  would  appear  that  places  are  localized  in  a  fine  spatial  resolution 
description  of  the  intensity  distribution. 

8.2.4  The  representation  of  fine  texture  detail 

A  series  of  computational  experiments  have  been  performed  to  examine  the  representa¬ 
tion  of  texture  detail.  Our  approach  has  been  to  first  study  the  description  of  texture  at 
the  smallest  available  visual  scale,  for  which  there  is  the  greatest  likelihood  that  the 
preceeding  stages  of  intensity  processing  are  linear  (larger  scales  may  be  mediated,  in 
part,  by  the  transient,  or  Y-cell,  channels,  which  are  known  to  be  highly  non-linear). 
For  the  smallest  available  scales,  therefore,  we  take  as  given  that  the  texture  description 
must  somehow  build  upon  a  difference-of-Gaussians  convolved  image.  The  currently- 
popular  hypothesis  is  that  zero-crossings  (ZCs)  in  the  convolved  image  serve  to  localize 
intensity  changes.  Part  of  our  task  has  been  to  examine  this  hypothesis  in  the  context 
of  dense,  fine-detail  texture,  for  which  virtually  no  attention  has  been  paid. 

In  this  regard,  one  problem  has  been  to  account  for  the  detection  of  the  orientation 
of  very  small  bars  and  blobs  at  the  limit  of  visual  resolution.  Consider  the  smallest  pro- 


posed  difference-of-Gaussians  channel  (w  =  1.3'  central  excitatory  region).  The  convolu¬ 
tion  of  a  short  line  segment  of  length  approximately  equal  to  u)  would  result  in  a  closed 
zero-crossing  contour  of  width  approximately  u>  and  a  fractionally  longer  length.  Detec¬ 
tion  of  the  orientation  of  this  small  bar  would  require  sensitivity  to  the  elongation  of  the 
closed  zero-crossing.  A  series  of  pilot  experiments  has  investigated  how  accutely  we 
measure  the  elongation  of  the  convolved  bars,  with  preliminary  results  suggesting  that 
we  are  sensitive  to  bar  orientation  where  the  closed  ZC  has  a  length-to-width  ratio  of 
only  1.4  to  1  or  less.  The  completion  of  these  experiments  awaits  the  installation  of  our 
digital  convolver. 

At  any  scale  of  convolution  the  position  of  a  zero  crossing  tends  to  meander, 
reflecting  smaller-scale  intensity  variations.  The  ZC  locus  at  any  given  scale  is  subject 
to  artifacts  that  cannot  be  resolved  at  that  scale,  such  as  failure  to  resolve  discrete  items 
having  separations  less  than  u/,  termination  effects  (blooming  of  the  ZCs  at  the  ends  of 
bars  or  lines),  and  displacement  of  ZCs  caused  by  adjacent  intensity  changes  within 
approximately  2u\  A  question  is  how  to  localize  an  intensity  change  at  the  smallest  scale 
in  the  face  of  these  artifacts.  We  have  found  phenomena  suggesting  that,  in  the  absence 
of  “supporting  evidence”  provided  by  spatial  coincidence  in  a  smaller  channel,  wc  use 
the  ZCs,  or  some  similar  basis  representation,  for  shape  interpretations. 

These  experiments  consist  of  observation  of  a  checkerboard  in  a  grey  surround. 
When  the  individual  squares  subtend  approximately  3-4'  the  squares  are  aggregated 
along  diagonals,  and  the  squares  along  the  border  are  prominent.  When  the  squares  sub¬ 
tend  approximately  2-3'  the  interior  is  a  fine  diagonal  mesh,  with  the  border  squares 
elongated.  When  they  subtend  only  1-2'  the  interior  is  a  uniform  grey,  the  border 
squares  appear  more  elongated,  and  the  corner  squares  are  more  prominent.  Finally,  at 


the  extreme  of  resolution,  at  approximately  .6-.8',  the  border  squares  are  barely  detect¬ 
able  while  the  corner  squares  are  sharp  pinpoints.  These  effects  are  all  reflected  in  the 
patterns  of  ZCs  in  the  correspondingly-scaled  convolutions.  Our  interpretation  is  that 
the  ZCs  at  the  smallest  available  scale  forms  the  basis  for  perceived  shape  of  texture 
detail,  despite  the  artifacts  of  the  convolution.  Also,  since  these  artifacts  persist  under 
scrutiny,  they  suggest  a  representation  of  smooth  contour  in  the  hyperacuity  range  (radii 
of  curvature  less  than  1 ' ). 

In  a  related  study  we  have  examined  the  ZCs  in  dense  texture  and  observed  that 
places  that  appear  prominent  —  to  act  as  place  tokens  for  apparent  one-dimensional 
organization  in  the  texture  —  seem  to  have  a  rather  lawful  interpretation  in  terms  of 
ZCs.  While  not  necessarily  corresponding  to  isolated  closed  ZCs,  they  appear  to  be  places 
of  high  combined  contrast  (slope  across  the  ZC)  and  curvature.  The  current  research  is 
investigating  various  neurophysiologically-piausible  mechanisms  for  localizing  very  small 
features  in  texture  on  the  basis  of  the  difference-of-Gaussians  convolved  image.  A  rea¬ 
sonable  starting  point  is  to  consider  ZCs,  but  evidence  from  other  studies  of  hyperacuity 
and  stereopsis  implicate  local  extrema  as  well. 
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